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ABSTRACT: 

PURPOSE: To provide an information input device as a human interface 
excellent in handleability by integrating pen input and speech input. 

CONSTITUTION: This information input device is equipped with an input pen 1 
which has a microphone la and a function for sending a speech signal gathered 
by the microphone la to a computer body through a nnr d or by radio, a display 
and tablet 2 which detects a touch by the input pen 1, means 5, 6, and 7 which 
performs a speech recognizing process and a natural language process for the 
speech signal from the microphone la, and means 8, 9, 10, and 11 which performs 
a character recognizing process, an image recognizing process, and a position 
recognizing process for the input signal from the table 2. Further, this 
device is equipped with a means 15 which combines the input from the microphone 
la and the input from the tablet 2 together and sends data or a command to an 
operating system 18 or application program 19. 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use o£ this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 

DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention relates to an information input unit for a user to input 
information, such as data and a command, in a personal computer, a word processor, etc. 
[0002] 

[Description of the Prior Art] As this kind of an information input device, although the keyboard is used 
from ancient times, recently, a tablet top is traced with a pen and the type which inputs an alphabetic 
character and a graphic form is also beginning to be used. Moreove r, what is depended on voice i nput is 
Re alizable by improvement in a speech recognition techniq ue! " 

looosf ' — 

[Problem(s) to be Solved by the Invention] Although various things were appearing as an informational 
input gestalt as mentioned above, it was used independently conventionally, respectively. For example, 
even if it was equipment equipped with the pen input ftinction and the voice input function, a pen input 
and voice input had been independent and the item inputted with a pen and the item inputted with voice 
were distinguished . Moreover, a mutual input was not related directly. 

[0004] Since the microphone for inputting voice is attached in the body of a computer on the other hand 
or it connected through the code, while wiring became obstructive, it was able to be used only near the 
body of a computer. Moreover, the power source of the body of a computer was switched on, and if it 
had not changed into the condition which can accept audio, voice input from a microphone was not able 
to be used. 

[0005] The place which this invention is proposed in view of the above-mentioned point, and is made 
into the purpose unites a pen input and voice input, and is to offer the information input imit as a user- 
friendly human interface. 
[0006] 

[Means for Solving the Problem] An input pen with the function toiransmit the sound signal extracted 
with a microphone and this microphone to the body of a computerlby the cable or wireless[i n order that 
this invention may attain the above-mentioned purpose. The tablerof the display combinaSon which 
detects contact of this input pen, ^idjjneans jo perform speech recognition proc essiag^ d natural 
language processing to the sound signarfromsaid microphone , A means to perfonn character 
recognition processmg, image recognition processing, and location recognition processing to the input 
signal from said tablet, He is trying to have a means to give data or a command to an operating system 
or an application program combining the input from said microphone, and the input from said tablet. 
[0007] For example, when the input from a microphone a nd the input from a tablet are inputs about the 
same object, t he higher one of whenever precognition J is adopted as the data given to an operating 
system ofan application program, or a command . Moreover, data.^6jiiade to input from a microphone 
^andjtjsmade to make editing^comiimids^ju ^^ in sertion and deletion, input gesture with an input 
pen. " ' — ^ 

[0008] It can have the memory which, on the other hand, accumulates the sound signal extracted with 
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the microphone in an input pen. 

[0009] Furthermore, a sound signal is transmitted from the body of a computer to an input pen, and 

sound can be reproduced with a microphone. 

[0010] 

[Function] If it is in the information input unit of this invention and a user talks toward the input pen^ 
which it had in the hand, the voice will be extracted with the microphone built in the input pen, and will 
je transmitted to the body of a computer by the cable or wireless . Moreover, the information will be 
mputted if the tablet of display combination is traced with an input pen. 

[0011] Inside the body of a computer, while speech recognition processing and natural language 
processing are performed to the sound signal from the nucrophone ot an input pen, c haracter recognition 
processing, image recognition processing, and location recognition processing are performed to the input 
signal from a tablet. 

[0012] And the input from a microphone and the input from a tablet are combined appropriately, and are 
given/asBata or a conmiand)to an operating system or an application program. For example, when the 
mput irom a microphone arid the input from a tablet a re inputs about the same object, th e higher one of 
whenever 44gco gnition J is adopted and it is given^'s data or a command^ Moreover, a user can input 
editing commmidsTsuch as insertion and deietion. wim an input pen to the data inputted from the 
micropnone. 

[0013] On the other hand, when it has the memory which accumulates the sound signal extracted with 

the microphone and there is no body of a computer in operating status, also when separating and 

working on the input pen from the body of a computer, voice can be inputted into it, and it can bundle 

up behind, and a sound signal can be transmitted to the body of a computer. 

[0014] Furthermore, a sound signal is transmitted from the body of a computer to an input pen, and 

sound can be reproduced with a microphone. 

[0015] 

[Example] Hereafter, with reference to a drawing, it explains about the example of this invention. 
[0016] Drawing 1 is the block diagram showing one example of the information input unit of this 
invention. 

[0017] While this example processes the signal from the input pen 1 which a user has and uses for a 
hand, the tablet 2 of the display combination which presses the input pen 1 and draws an alphabetic 
character, a graphic form, etc., and the input pen 1 and a tablet 2 in drawing 1 Two or more circuit 
sections which give these a signal or control (both hardware and software are included.) It consists of an 
operating system 18 which manages the overall confrol within the body of a computer, and an 
application program 19 which does a predetermined activity. 

[0018] Moreover, microphone la for extracting the voice other than the ftmdamental fimction to trace a 
tablet 2 on the input pen 1, and to give the locus as information to it. While transmitting the sound signal 
which was extracted by memory lb which accumulates the sound signal exfracted by this microphone 
la, and microphone la, or was accumulated by memory lb to the body of a computer Transceiver 
section Ic^ which receives the sound signal at the time of using a microphone 1 as a loudspeaker from the 
body oTacomputer is prepared. In addition, memory lb may be accumulated by which method of digital 
one or an analog. Moreover, it is possible between the input pen 1, and a receive section 3 and the 
transmitting section 4 not only wireless but to connect with a cable. 

[0019] As two or more circuit sections which give these a signal or control on the other hand while 
processing the signal from the input pen 1 and a tablet 2 The part which divides roughly and 
communicates with the input pen 1 (a receive section 3 , transmitting section 4), The part which extracts 
an alphabetic character and a comma nd from th e inputted voice (the speech recognition section S^^ tiie 
n atural-language-proce s sing section o, an aipn abet ic character and a command input area 7) , The part 
which extracts an alphabetic character, an image, a location, etc. from the locus of contact of the input 
pen 1 to a tablet 2 (the image-processing section 8, the character recognition section 9, the image 
recognition section 10, the gesture input section 1 1, the location recognition section 12, the location 
input section 13, image input section 14), It consists of a part (voice generation section 16) which 
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generates the sound signal given to the input pen 1, a part (display and control section 17) which 
controls the d isplay of a tablet 2, and a part (comprehensive processing section 15) which performs the 
whole control, in aaaition, the natural-language-processing section 6, and an alphabetic character and a 
command input area 7 are used also for the processing to the hand written character inputted from the 
tablet 2. 

[0020] Next, actuation of the above-mentioned example is explained. 

[0021] First, the user shall have started the suitable application program 19, in order to be in the 
condition in which an operating system 18 and a dialogue are possible or to carry out desired business. 
An application program 19 is a word processor etc. 

[0022] Although a user inputs data or a command according to the function of an operating system 18 or 
an appUcation program 19, voice input and a pen input can be used as the approach of an input ofd ata or 
a command in this case. In addition, when the keyboard is connected to the body ot a computer, 
naturally the input from a keyboard can also be performed. 

[0023] If a user speaks toward the input pen 1, t his voice will be extracted by microphone la and will be 
transmitted to the body side of a computer by transceiver ^eclion Ic by w ireless, m adQition, by ' 
^onsidenng as sound recording mode, ttie sound signal is temporanly accumulated in memory lb, and it 
can transmit to the body side of a computer collectively behind. In this case, as long as there is an input 
pen 1, it is not necessary to be near the body of a computer at the time of utterance and, and the body of 
a computer may not be in operating status. 

[0024] In the body side of a computer, by the receive section 3. the sound signal from the input pen 1 is 
receive37 it gets over, and the speech recognition secti on 5 is passed. 

[0025] In the speech recognition section X to the received so un d signal^ well-known analy sis^ 
pro cessings (voice section detection, so na gaphy, afeature extraction, segmentation, phoneme 
recogiiition, etc.) are performed, and the data in which the train of a phoneme is shown are outputted . 
[0026] In th e natural-language-processing section 6, to the train of a phoneme, well-known analysis 
processings [word collating, syntax analysis, semantic analysis, etc.) are p erformed in the field of 
machine translation, and the extracted character stnng is outputted. " 

[0027] In an alphabetic character and a command input area 7, it prepares to a predet ermined format by 
making into data or a command the character stri n g^given from the natural-language-processmg section 
6, and g ives the c omprehensive p roce ssing section 15. 

[0028 ] In the comprehensive processing section 15, the data or the command given from the alphabetic 
chamctfiLai id the command input area 7 is given to an operating system 18. and if an operating system 
18 is a command to self, it will accept it, and it hands over the thing to an application program 19 to an 

I application program 19. ' ~" ' ~" ~~ 

^ [0029] On the other hand, when a user traces a tablet 2 top and draws an alphabetic character, a graphic 
form, etc. with the input pen 1, the condition of the contact to a tablet 2 is detected by the image- 
processing section 8. 

[0030] In the character recognition section 9, when it distinguishes whether what was drawn is an 
alphabetic character and there is an alphabetic character which can be recognized about it, the alphabetic 
character recognized m the natural-language-processing section 6 is passed. Actuation of the natural- 
language-processing section 6 to henceforth is the same as that of the case of voice input. 
[0031] Moreover, in the image recognition section 10, when it distinguishes whether it is the gesture 
(motion of the pen specified corresponding to predetermined commands, such as insertion and deletion.) 

f which was drawn on the tablet 2 and there is gesture which can be recognized about it, it is told to the 
gesture input secfion 11. 
[0032] In the gesture input section 1 1, the data or the command corresponding to gesture is given to the 
comprehensive processing section 15. 

[0033] In the comprehensive processing section 15, the data or the command given from the gesture 
input section 7 is given to an operating system 18, and if an operating system 18 is the data and the 
command to an application program 19, it will be handed over to an application program 19. 
[0034] Moreover, in the location recognition section 12, the contact location of the input pen on a tablet 
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2 is recognized, and the information is given to the location input section 13. 

[0035] The location input section 13 prepares the data in which a location is shown to a predetermined 
format, and passes them to the comprehensive processing section 15. 

[0036] In the comprehensive processing section 15, the data given from the location input section 13 are 
given to an operating system 18, and if an operating system 18 is data to an application program 19, it 
will be handed over to an application program 19. 

[0037] Moreover, the image input section 14 is given to the comprehensive processing section 15 by 
making into image data the locus itself drawn on the tablet 2, and the comprehensive processing section 
15 is given to an operating system 18, and it hands it over to an application program 19 about the data to 
an application program 19. 

[0038] On the other hand, when operating microphone la of the input pen 1 as a loudspeaker, the 
comprehensive processing section 15 gives audio data to the voice generation section 16. 
[0039] In the voice generation section 16, speech synthesis etc. is performed, a sound signal is 
generated, it gives the transmitting section 4, and the transmitting section 4 transmits to the input pen 1 
by wireless. 

[0040] In the input pen 1 side, if transceiver section Ic receives this, it will get over, will consider as a 
sound signal, it will give microphone la, and will change into soimd. 

[0041] Moreover, a display and control section 17 controls the contents displayed on the basis of control 
of the comprehensive processing section 15, and the tablet 2 used as display combination. 
[0042] Although basic actuation of voice input and a pen input has been explained until now, in fact, 
both inputs are related closely mutually and performed. Namely, the comprehensive processing section 
15 combines appropriately the input from microphone la of the input pen 1, and the input from a tablet 
2, and gives them to an operating system 18 and an application program 19 as data or a command. 
[0043] For example, when the input from microphone la and the input from a tablet 2 are inputs about 
the same object, the comprehensive processing section 15 adopts the higher one of whenever 
[ recognition ] as the data given to an operating system 18 or an application program 19, or a command 
(when the data input of an item with an applicat ionprogram 1 9 is being demanded as an ex ample etc.). 
In addition, whenever [ recognition ] is a value extractable m the process of processing of the 
recognition in the speech recognition section 5, the natural-language-processing section 6, the character 
recognition section 9, and image recognition section 10 grade. By doing in this way, the precision of 
recognition can be raised sharply. 

[0044] Moreover, a text etc. can be smoothly inputted by making editing commands, such as insertion 
and deletion, input by gesture with the input pen 1 and a tablet 2, inputting data from microphone la. 
[0045] For example, although "BTA" was inputted with voice from microphone la, the input pen 1 
performs the gesture oftEe aeietio n wnicn draw s "x" on a mistaken part "B" o n a tablet 2 to input 
^DATA" (**) and originally, as shown m drawmg Z ^**)- ' ' ~ 

[0046] Thereby, "B" is eliminated and " l A" ot one character is got blocked in front (**). 
[0047] Subsequently, it will be set to "DATA", if the gesture of insertion is performed and "DA" is 
inputted with voice from (**) and microphone la before "TA" (**). 

[0048] In addition, it supposes that a back alphabetic character remains as it is, and when an alphabetic 
character is newly inserted, you may make it a back alphabetic character shift only by making the 
deleted alphabetic character into a null if needed, although he was trying to get a back alphabetic 
^character blocked with drawing 2 in front in the case of deletion. 
[0049] 

[Effect of the Invention] If it is in the information input unit of this invention as explained above, there 
is the following effectiveness. 

[0050] ** Since the input of data or a command can be performed combining voice input and a pen 
input appropriately, the user-friendliness as a human interface becomes very good. 
_ [0051] ** Even if it makes it voice input and makes it a pen input, there is an inadequate point in respect 
of recognition precision, but since whenever [ recognition ] adopts the higher one when there is an input 
from both sides about the same object, recognition precision can be raised sharply. 
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[0052] ** Since editing commands, such as insertion and deletion, can be inputted by the gesture of an 

input pen, inputting data with voice, smooth document preparation etc. can be performed . 

[0053] ** Since the microphone and the body of a computer of an input pen are connectable on radio, 

there is no obstructive wiring and it can consider as comfortable work environment. 

[0054] ** If the input pen is carried when there is no body of a computer in operating status, or when 

separating and working from the body of a computer since the memory which accumulates a sound 

signal is prepared in the input pen, it is not necessary to input document data etc. on that spot, to bimdle 

up behind, to transmit to the body of a computer, and to forget the idea thought of. 

[Translation done.] 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 


CLAIMS 


[Claim(s)] 

[Claim 1] An ij^ul pen with the function to transmit the sound signal extracted with ajmcroghoneand 
this microphone to the body of a computer by the cable or wireless, The tablet of the display 
combination which detents rpntact of this in put pen , and a means to perform speech recogni tion 
processing a n d natural language processing to the sound signal from said microphone. A means to 
perform character recognition processing, image recognition processing, and location recognition 
processing to the input signal from said tablet, The information input unit characterized by having a 
means to give data or a command to an operating system or an application program combining the input 
from said microphone, and the input from said tablet. 

[Claim 2] The information input unit according to claim 1 characterized by adopting the higher one of 
whenever [ recognition ] as the data given to an operating system or an application program, or a 
command when the input from a microphone and the input from a tablet were inputs about the same 
object. 

[Claim 3] The information input unit according to claim 1 or 2 which was made to mgutjiata-from^ 
microphone and was characterized by making editin g commands, such as insertion and deletion, input 
by gesture with the input pen. 

[Claim 4] The information input unit according to claim 1, 2, or 3 characterized by having the memory 
which accumulates the sound signal extracted with the microphone in an input pen. 
[Claim 5] The information input unit according to claim 1, 2, 3, or 4 characterized by transmitting a 
sound signal from the body of a computer to an input pen, and reproducing sound with a microphone. 


[Translation done.] 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use o£ this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3, In the drawings, any words are not translated. 


DESCRIPTION OF DRAWINGS 


[Brief Description of the Drawings] 

[Drawing 1] It is the block diagram showing one example of the information input unit of this invention. 

[Drawing 2] It is drawing showing the situation of edit by the voice input and the gesture of an 
alphabetic character. 
[Description of Notations] 

1 .... Input pen 
v/la — Microphone 

lb " Memory 
\^ Ic — Transceiver section 

2 .... Tablet 

^3 .... Receive section 
V 4 .... Transmitting section 


(^6 .... Natural-language-processing section 

7 .... An alphabetic character and command input area 

8 .... Image-processing section 

9 .... Character recognition section 

10 ~ Image recognition section 
11— Gesture input section 

12 ~ Location recognition section 

13 — Location input section 

14 — Image input section 

15 — Comprehensive processing section 

16 — Voice generation section 
\ n - Dispk :iLaru i control sgction 

18 - Operating system 

19 ~ Application program 


[Translation done.] 


http://www4.ipdl.ncipi.go.jp/cgi-bin/tran_web_cgi_ejje 4/6/06 



